Skip to content

SARIF Output Format#4

Open
MikeeBuilds wants to merge 8 commits intomainfrom
auto-claude/009-sarif-output-format
Open

SARIF Output Format#4
MikeeBuilds wants to merge 8 commits intomainfrom
auto-claude/009-sarif-output-format

Conversation

@MikeeBuilds
Copy link
Owner

Add a --sarif flag that outputs scan results in SARIF (Static Analysis Results Interchange Format) v2.1.0. This enables direct integration with GitHub Advanced Security, Azure DevOps, and other SARIF-compatible code scanning platforms. Findings map to SARIF result objects with rule metadata, severity levels, and remediation guidance.

@gemini-code-assist
Copy link

Summary of Changes

Hello @MikeeBuilds, I'm Gemini Code Assist1! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request significantly enhances the clawpinch tool by adding support for SARIF (Static Analysis Results Interchange Format) output. This new capability allows clawpinch scan results to be directly consumed by SARIF-compatible platforms such as GitHub Advanced Security and Azure DevOps, facilitating automated security analysis, inline code annotations in pull requests, and centralized vulnerability management within development workflows.

Highlights

  • SARIF Output Integration: Introduced a new --sarif command-line flag to enable output of scan results in SARIF v2.1.0 format, facilitating integration with code scanning platforms.
  • GitHub Actions Documentation: Added a detailed guide (docs/github-actions-sarif.md) on integrating ClawPinch with GitHub Code Scanning, including workflow examples and troubleshooting steps.
  • SARIF Conversion Logic: Implemented a new helper script (scripts/helpers/sarif.sh) responsible for transforming ClawPinch's internal findings JSON into the standardized SARIF format, including severity mapping and rule metadata.
  • Output Prioritization: Modified the main script to prioritize SARIF output when the --sarif flag is used, ensuring proper formatting for external tools and suppressing standard UI.
Changelog
  • clawpinch.sh
    • Added source "$HELPERS_DIR/sarif.sh" to include the new SARIF helper script.
    • Introduced SARIF_OUTPUT=0 default and --sarif option parsing to set SARIF_OUTPUT=1.
    • Exported SARIF_OUTPUT for use in helper scripts.
    • Updated conditional checks for banner and scanner output to suppress standard UI when SARIF_OUTPUT is enabled.
    • Modified the final output logic to call convert_to_sarif when SARIF_OUTPUT is set, preceding JSON output.
  • docs/github-actions-sarif.md
    • New file created.
    • Provides a comprehensive guide for integrating ClawPinch with GitHub Actions for code scanning.
    • Details quick start, how SARIF output works, uploading results, viewing findings in PRs and the Security tab.
    • Includes advanced configurations for specific directories, deep scans, failing builds on critical findings, and multiple SARIF uploads.
    • Explains how to combine ClawPinch with other scanners and offers troubleshooting tips for common SARIF integration issues.
    • References the SARIF output format structure and severity mapping.
    • Presents a complete CI/CD pipeline example for enforcing security gates.
  • scripts/helpers/sarif.sh
    • New file created.
    • Contains the convert_to_sarif bash function to generate SARIF v2.1.0 JSON from ClawPinch findings.
    • Includes logic to map ClawPinch severity levels (critical, warn, info) to SARIF levels (error, warning, note).
    • Dynamically extracts tool version from package.json.
    • Structures SARIF output with tool driver information, rules (including ID, title, description, help URI, and category properties), and results (including rule ID, level, message with markdown, evidence, and auto_fix properties).
    • Requires jq for JSON processing.
Activity
  • No specific human activity (comments, reviews, approvals) has been recorded for this pull request yet. The changes primarily focus on initial feature implementation and documentation.
Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature Command Description
Code Review /gemini review Performs a code review for the current pull request in its current state.
Pull Request Summary /gemini summary Provides a summary of the current pull request in its current state.
Comment @gemini-code-assist Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help /gemini help Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for Github and other Google products, sign up here.

You can also get AI-powered code generation, chat, as well as code reviews directly in the IDE at no cost with the Gemini Code Assist IDE Extension.

Footnotes

  1. Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces SARIF output, a valuable feature for integrating with code scanning platforms like GitHub Advanced Security. The implementation is well-structured, including a dedicated helper script for SARIF conversion and comprehensive documentation. My review focuses on a few areas for improvement: removing an unnecessary variable export, refactoring duplicated logic for better maintainability, cleaning up some dead code, and aligning the SARIF output with the provided documentation.

Comment on lines +103 to +129
"results": (
. | map({
"ruleId": .id,
"level": (
if .severity == "critical" then "error"
elif .severity == "warn" then "warning"
elif .severity == "info" then "note"
elif .severity == "ok" then "none"
else "note"
end
),
"message": {
"text": .title,
"markdown": (
if .remediation != "" then
(.description + "\n\n**Remediation:** " + .remediation)
else
.description
end
)
},
"properties": {
"evidence": .evidence,
"auto_fix": .auto_fix
}
})
)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current jq query includes findings with ok severity in the SARIF output, mapping their level to none. This contradicts the documentation in docs/github-actions-sarif.md, which states: "ok → Not included in SARIF output (only findings)".

To align the implementation with the documentation and produce a cleaner SARIF file, you should filter out these ok findings entirely. This also allows for simplifying the level-mapping logic within the query.

          "results": [
            .[] | select(.severity != "ok") | {
              "ruleId": .id,
              "level": (
                if .severity == "critical" then "error"
                elif .severity == "warn" then "warning"
                else "note"
                end
              ),
              "message": {
                "text": .title,
                "markdown": (
                  if .remediation != "" then
                    (.description + "\n\n**Remediation:** " + .remediation)
                  else
                    .description
                  end
                )
              },
              "properties": {
                "evidence": .evidence,
                "auto_fix": .auto_fix
              }
            }
          ]

clawpinch.sh Outdated
export CLAWPINCH_DEEP="$DEEP"
export CLAWPINCH_SHOW_FIX="$SHOW_FIX"
export CLAWPINCH_CONFIG_DIR="$CONFIG_DIR"
export SARIF_OUTPUT

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The SARIF_OUTPUT variable is used only within this main script to control output flow and does not need to be exported. Exporting variables that are not needed by child processes can pollute their environment and potentially lead to unexpected behavior if a child script uses a variable with the same name for a different purpose.

Suggested change
export SARIF_OUTPUT
# SARIF_OUTPUT is used only in this script and does not need to be exported.

clawpinch.sh Outdated
# ─── Banner ──────────────────────────────────────────────────────────────────

if [[ "$JSON_OUTPUT" -eq 0 ]] && [[ "$QUIET" -eq 0 ]]; then
if [[ "$JSON_OUTPUT" -eq 0 ]] && [[ "$SARIF_OUTPUT" -eq 0 ]] && [[ "$QUIET" -eq 0 ]]; then

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This condition [[ "$JSON_OUTPUT" -eq 0 ]] && [[ "$SARIF_OUTPUT" -eq 0 ]] && [[ "$QUIET" -eq 0 ]] is repeated 5 times in the script (here and on lines 159, 180, 205, 214). To improve readability and maintainability, consider refactoring this into a helper function or a single variable.

For example, you could define a variable after argument parsing:

# Determine if we should print the rich terminal UI
local should_print_ui=1
if [[ "$JSON_OUTPUT" -eq 1 || "$SARIF_OUTPUT" -eq 1 || "$QUIET" -eq 1 ]]; then
  should_print_ui=0
fi

Then you could replace all occurrences of the long condition with a simpler if [[ "$should_print_ui" -eq 1 ]]; then. This would make the code cleaner and easier to modify in the future.

Comment on lines +17 to +25
_sarif_level() {
case "$1" in
critical) echo "error" ;;
warn) echo "warning" ;;
info) echo "note" ;;
ok) echo "none" ;;
*) echo "note" ;;
esac
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This _sarif_level shell function is defined but never used. The mapping from the tool's severity levels to SARIF levels is handled directly inside the jq query on lines 106-112. This function is dead code and should be removed to keep the script clean.

@greptile-apps
Copy link

greptile-apps bot commented Feb 7, 2026

Greptile Overview

Greptile Summary

This PR adds a --sarif output mode to clawpinch.sh, routing the aggregated scan findings through a new scripts/helpers/sarif.sh converter that emits SARIF v2.1.0 for GitHub Code Scanning and similar platforms. It also introduces a GitHub Actions guide showing how to run ClawPinch in CI and upload the generated SARIF.

Main integration point is clawpinch.sh switching its output path: when --sarif is set it suppresses the interactive/TTY UI and prints SARIF to stdout for redirection into a .sarif file, which can then be uploaded via github/codeql-action/upload-sarif.

Issues needing attention before merge:

  • scripts/helpers/sarif.sh should be made null-safe for optional finding fields (evidence, auto_fix, remediation) to avoid emitting null into SARIF properties/markdown, which can fail validation in downstream tooling.
  • The docs’ example $schema URL references master and should be pinned (or omitted) to avoid non-deterministic validator behavior.

Confidence Score: 4/5

  • Mostly safe to merge, but fix SARIF field normalization to avoid invalid output in some inputs.
  • The core wiring for --sarif is straightforward and localized, and the SARIF converter generally follows the expected structure. The remaining concern is correctness/compatibility when optional finding fields are null, which can break SARIF validation or ingestion for certain scanners/inputs.
  • scripts/helpers/sarif.sh

Important Files Changed

Filename Overview
clawpinch.sh Adds --sarif flag and suppresses terminal UI when emitting SARIF; output routing looks consistent with existing JSON/quiet handling.
scripts/helpers/sarif.sh Introduces jq-based SARIF v2.1.0 conversion; needs null-safety/normalization for optional fields to avoid invalid SARIF JSON.
docs/github-actions-sarif.md Adds GitHub Actions integration guide for SARIF uploads; example schema URL should be pinned for determinism.

Sequence Diagram

sequenceDiagram
  participant U as User/CI
  participant CP as clawpinch.sh
  participant SC as Scanners
  participant S as sarif.sh
  participant JQ as jq
  participant GH as upload-sarif

  U->>CP: Run with --sarif
  CP->>SC: Execute scanners
  SC-->>CP: Findings JSON arrays
  CP->>JQ: Sort/filter findings JSON
  JQ-->>CP: SORTED_FINDINGS
  CP->>S: convert_to_sarif(SORTED_FINDINGS)
  S->>JQ: Build SARIF document
  JQ-->>S: SARIF JSON
  S-->>CP: SARIF JSON (stdout)
  CP-->>U: SARIF JSON output
  U->>GH: Upload SARIF file
  GH-->>U: Code Scanning alerts
Loading

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines +103 to +129
"results": (
. | map({
"ruleId": .id,
"level": (
if .severity == "critical" then "error"
elif .severity == "warn" then "warning"
elif .severity == "info" then "note"
elif .severity == "ok" then "none"
else "note"
end
),
"message": {
"text": .title,
"markdown": (
if .remediation != "" then
(.description + "\n\n**Remediation:** " + .remediation)
else
.description
end
)
},
"properties": {
"evidence": .evidence,
"auto_fix": .auto_fix
}
})
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SARIF results missing locations field - GitHub Code Scanning won't show inline file annotations

Current output only includes messages without file/line mappings. For GitHub to display findings inline on specific files, each result needs a locations array:

Suggested change
"results": (
. | map({
"ruleId": .id,
"level": (
if .severity == "critical" then "error"
elif .severity == "warn" then "warning"
elif .severity == "info" then "note"
elif .severity == "ok" then "none"
else "note"
end
),
"message": {
"text": .title,
"markdown": (
if .remediation != "" then
(.description + "\n\n**Remediation:** " + .remediation)
else
.description
end
)
},
"properties": {
"evidence": .evidence,
"auto_fix": .auto_fix
}
})
)
"results": (
. | map({
"ruleId": .id,
"level": (
if .severity == "critical" then "error"
elif .severity == "warn" then "warning"
elif .severity == "info" then "note"
elif .severity == "ok" then "none"
else "note"
end
),
"message": {
"text": .title,
"markdown": (
if .remediation != "" then
(.description + "\n\n**Remediation:** " + .remediation)
else
.description
end
)
},
"locations": [
{
"physicalLocation": {
"artifactLocation": {
"uri": "."
}
}
}
],
"properties": {
"evidence": .evidence,
"auto_fix": .auto_fix
}
})
)

This maps all findings to the repository root. For more precise annotations, ClawPinch findings would need to include file paths (e.g., from $OPENCLAW_CONFIG evidence).

Prompt To Fix With AI
This is a comment left during a code review.
Path: scripts/helpers/sarif.sh
Line: 103:129

Comment:
SARIF results missing `locations` field - GitHub Code Scanning won't show inline file annotations

Current output only includes messages without file/line mappings. For GitHub to display findings inline on specific files, each result needs a `locations` array:

```suggestion
          "results": (
            . | map({
              "ruleId": .id,
              "level": (
                if .severity == "critical" then "error"
                elif .severity == "warn" then "warning"
                elif .severity == "info" then "note"
                elif .severity == "ok" then "none"
                else "note"
                end
              ),
              "message": {
                "text": .title,
                "markdown": (
                  if .remediation != "" then
                    (.description + "\n\n**Remediation:** " + .remediation)
                  else
                    .description
                  end
                )
              },
              "locations": [
                {
                  "physicalLocation": {
                    "artifactLocation": {
                      "uri": "."
                    }
                  }
                }
              ],
              "properties": {
                "evidence": .evidence,
                "auto_fix": .auto_fix
              }
            })
          )
```

This maps all findings to the repository root. For more precise annotations, ClawPinch findings would need to include file paths (e.g., from `$OPENCLAW_CONFIG` evidence).

How can I resolve this? If you propose a fix, please make it concise.

@MikeeBuilds MikeeBuilds force-pushed the auto-claude/009-sarif-output-format branch from a16ea71 to 0fffd6b Compare February 9, 2026 02:17
…, add locations, refactor UI condition

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@MikeeBuilds
Copy link
Owner Author

All review feedback has been addressed:

  • Removed dead _sarif_level() function
  • Filtered out ok severity findings from SARIF output
  • Added locations array to SARIF results
  • Removed export SARIF_OUTPUT (not needed by subprocesses)
  • Refactored repeated UI condition into _SHOW_UI variable

@gemini-code-assist @greptile-apps — requesting re-review. Thanks!

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces SARIF output format for scan results, enabling integration with tools like GitHub Advanced Security. The changes include adding a --sarif flag, a new helper script for SARIF conversion, and comprehensive documentation for GitHub Actions integration. The implementation is well-structured, using a dedicated script for formatting and refactoring the main script to handle different output modes cleanly. My review focuses on improving the generated SARIF output for better usability. I've identified that the help URIs for rules are incorrect, and I've provided suggestions to fix them in both the generation script and the documentation.

fullDescription: {
text: .description
},
helpUri: ($tool_uri + "#" + (.id | ascii_downcase)),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The generated helpUri points to the main repository page, which doesn't have anchors for check IDs, resulting in a broken link. It should point to the check-catalog.md file where the check details are documented.

To make this link work correctly, you would also need to add HTML anchors to the headers in references/check-catalog.md, for example: ### <a name="chk-cfg-001"></a>CHK-CFG-001 -- ....

This change updates the URI to point to the correct file. The anchor will work if you add the corresponding lowercase name attributes to the headers in the catalog.

Suggested change
helpUri: ($tool_uri + "#" + (.id | ascii_downcase)),
helpUri: ($tool_uri + "/blob/main/references/check-catalog.md#" + (.id | ascii_downcase)),

"shortDescription": {
"text": "Gateway listening on 0.0.0.0"
},
"helpUri": "https://github.com/MikeeBuilds/clawpinch#chk-cfg-001",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example helpUri here points to a non-existent anchor on the main repository page. This should be updated to reflect the correct link to the check documentation in references/check-catalog.md. This will prevent confusion for users looking at the example SARIF output and will align with the fix in sarif.sh.

Suggested change
"helpUri": "https://github.com/MikeeBuilds/clawpinch#chk-cfg-001",
"helpUri": "https://github.com/MikeeBuilds/clawpinch/blob/main/references/check-catalog.md#chk-cfg-001",

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

Comment on lines +62 to +88
. | [.[] | select(.severity != "ok")] | map({
id: .id,
name: .title,
shortDescription: {
text: .title
},
fullDescription: {
text: .description
},
helpUri: ($tool_uri + "#" + (.id | ascii_downcase)),
properties: {
category: (
if .id | startswith("CHK-CFG-") then "configuration"
elif .id | startswith("CHK-SEC-") then "secrets"
elif .id | startswith("CHK-NET-") then "network"
elif .id | startswith("CHK-SKL-") then "skills"
elif .id | startswith("CHK-PRM-") then "permissions"
elif .id | startswith("CHK-CRN-") then "cron"
elif .id | startswith("CHK-CVE-") then "cve"
elif .id | startswith("CHK-SUP-") then "supply-chain"
else "general"
end
)
}
})
| unique_by(.id)
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rules missing defaultConfiguration.level field

The SARIF spec and your own documentation (line 308-310 in docs/github-actions-sarif.md) show that rules should include defaultConfiguration.level, but this implementation doesn't include it. While not strictly required by SARIF v2.1.0, this field helps tools like GitHub Code Scanning understand the default severity of each rule.

Suggested change
. | [.[] | select(.severity != "ok")] | map({
id: .id,
name: .title,
shortDescription: {
text: .title
},
fullDescription: {
text: .description
},
helpUri: ($tool_uri + "#" + (.id | ascii_downcase)),
properties: {
category: (
if .id | startswith("CHK-CFG-") then "configuration"
elif .id | startswith("CHK-SEC-") then "secrets"
elif .id | startswith("CHK-NET-") then "network"
elif .id | startswith("CHK-SKL-") then "skills"
elif .id | startswith("CHK-PRM-") then "permissions"
elif .id | startswith("CHK-CRN-") then "cron"
elif .id | startswith("CHK-CVE-") then "cve"
elif .id | startswith("CHK-SUP-") then "supply-chain"
else "general"
end
)
}
})
| unique_by(.id)
)
"rules": (
. | [.[] | select(.severity != "ok")] | map({
id: .id,
name: .title,
shortDescription: {
text: .title
},
fullDescription: {
text: .description
},
helpUri: ($tool_uri + "#" + (.id | ascii_downcase)),
defaultConfiguration: {
level: (
if .severity == "critical" then "error"
elif .severity == "warn" then "warning"
elif .severity == "info" then "note"
else "note"
end
)
},
properties: {
category: (
if .id | startswith("CHK-CFG-") then "configuration"
elif .id | startswith("CHK-SEC-") then "secrets"
elif .id | startswith("CHK-NET-") then "network"
elif .id | startswith("CHK-SKL-") then "skills"
elif .id | startswith("CHK-PRM-") then "permissions"
elif .id | startswith("CHK-CRN-") then "cron"
elif .id | startswith("CHK-CVE-") then "cve"
elif .id | startsWith("CHK-SUP-") then "supply-chain"
else "general"
end
)
}
})
| unique_by(.id)
)

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: scripts/helpers/sarif.sh
Line: 62:88

Comment:
Rules missing `defaultConfiguration.level` field

The SARIF spec and your own documentation (line 308-310 in `docs/github-actions-sarif.md`) show that rules should include `defaultConfiguration.level`, but this implementation doesn't include it. While not strictly required by SARIF v2.1.0, this field helps tools like GitHub Code Scanning understand the default severity of each rule.

```suggestion
              "rules": (
                . | [.[] | select(.severity != "ok")] | map({
                  id: .id,
                  name: .title,
                  shortDescription: {
                    text: .title
                  },
                  fullDescription: {
                    text: .description
                  },
                  helpUri: ($tool_uri + "#" + (.id | ascii_downcase)),
                  defaultConfiguration: {
                    level: (
                      if .severity == "critical" then "error"
                      elif .severity == "warn" then "warning"
                      elif .severity == "info" then "note"
                      else "note"
                      end
                    )
                  },
                  properties: {
                    category: (
                      if .id | startswith("CHK-CFG-") then "configuration"
                      elif .id | startswith("CHK-SEC-") then "secrets"
                      elif .id | startswith("CHK-NET-") then "network"
                      elif .id | startswith("CHK-SKL-") then "skills"
                      elif .id | startswith("CHK-PRM-") then "permissions"
                      elif .id | startswith("CHK-CRN-") then "cron"
                      elif .id | startswith("CHK-CVE-") then "cve"
                      elif .id | startsWith("CHK-SUP-") then "supply-chain"
                      else "general"
                      end
                    )
                  }
                })
                | unique_by(.id)
              )
```

<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

…evel to SARIF rules

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@MikeeBuilds
Copy link
Owner Author

Round 2 feedback addressed:

  • Fixed helpUri to point to references/check-catalog.md#<id> instead of the repo root (both in sarif.sh and docs/github-actions-sarif.md example)
  • Added defaultConfiguration.level to SARIF rules with proper severity mapping

@gemini-code-assist @greptile-apps — requesting re-review. Thanks!

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces SARIF output, a valuable feature for integrating with security dashboards in platforms like GitHub. The implementation is well-structured, adding a dedicated helper script for SARIF conversion and comprehensive documentation with excellent GitHub Actions examples. My review includes a fix for a bug that generates broken help links in the SARIF report, suggestions to resolve version inconsistencies, and recommendations to make the documentation examples more robust. Overall, this is a strong contribution.

fullDescription: {
text: .description
},
helpUri: ($tool_uri + "/blob/main/references/check-catalog.md#" + (.id | ascii_downcase)),

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The current jq expression to generate the helpUri anchor (.id | ascii_downcase) will result in broken links. GitHub's anchor generation for markdown headers is more complex (e.g., for ### CHK-CFG-001 -- ..., the anchor is #chk-cfg-001----...). Linking to the document itself is a more robust solution than generating a broken fragment link, allowing users to search for the check ID on the page.

Suggested change
helpUri: ($tool_uri + "/blob/main/references/check-catalog.md#" + (.id | ascii_downcase)),
helpUri: ($tool_uri + "/blob/main/references/check-catalog.md"),

Comment on lines +31 to +36
local tool_version="1.2.0"
local package_json
package_json="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/package.json"
if [[ -f "$package_json" ]]; then
tool_version="$(jq -r '.version // "1.2.0"' "$package_json" 2>/dev/null || echo "1.2.0")"
fi

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The fallback version 1.2.0 is hardcoded here and is inconsistent with the version in package.json (1.2.1). This will cause an incorrect tool version to be reported in the SARIF file if reading package.json fails. Please update this to 1.2.1 for consistency.

Suggested change
local tool_version="1.2.0"
local package_json
package_json="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/package.json"
if [[ -f "$package_json" ]]; then
tool_version="$(jq -r '.version // "1.2.0"' "$package_json" 2>/dev/null || echo "1.2.0")"
fi
# Get tool version from package.json (fallback to 1.2.1)
local tool_version="1.2.1"
local package_json
package_json="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/package.json"
if [[ -f "$package_json" ]]; then
tool_version="$(jq -r '.version // "1.2.1"' "$package_json" 2>/dev/null || echo "1.2.1")"
fi

npx clawpinch --sarif --no-interactive > clawpinch.sarif

# Check if any critical findings exist
if jq -e '.runs[0].results[] | select(.level == "error")' clawpinch.sarif > /dev/null; then

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To make this example more robust, consider using optional chaining (?) in the jq command. This prevents the step from failing if the SARIF file is valid but doesn't contain a results array (e.g., if no findings were produced), which is a good practice for CI workflows.

Suggested change
if jq -e '.runs[0].results[] | select(.level == "error")' clawpinch.sarif > /dev/null; then
if jq -e '.runs[0]?.results[]? | select(.level == "error")' clawpinch.sarif > /dev/null; then

"tool": {
"driver": {
"name": "ClawPinch",
"version": "1.2.0",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The tool version in this example SARIF output is 1.2.0, but the current version in package.json is 1.2.1. Please update the example to reflect the correct version for consistency.

Suggested change
"version": "1.2.0",
"version": "1.2.1",

npx clawpinch --sarif --no-interactive > clawpinch.sarif

# Count critical findings
CRITICAL=$(jq '[.runs[0].results[] | select(.level == "error")] | length' clawpinch.sarif)

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

To make this example more robust, consider using optional chaining (?) in the jq command. This will prevent the step from failing if the SARIF file is valid but contains no results array, for example, if no findings were produced.

Suggested change
CRITICAL=$(jq '[.runs[0].results[] | select(.level == "error")] | length' clawpinch.sarif)
CRITICAL=$(jq '[.runs[0]?.results[]? | select(.level == "error")] | length' clawpinch.sarif)

…chaining

- Update hardcoded version fallback from 1.2.0 to 1.2.1 to match package.json
- Remove broken ascii_downcase fragment anchor from helpUri, point to
  check-catalog.md without fragment
- Use jq optional chaining (.runs[0]?.results[]?) in CI examples for
  robustness when SARIF has empty/missing runs
- Update example SARIF version and helpUri in docs to match code changes
- Greptile comment #7 (defaultConfiguration.level) already addressed in round 2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@MikeeBuilds
Copy link
Owner Author

Round 3 review feedback addressed in 4a83c29:

  1. sarif.sh:36 — Updated hardcoded version fallback from 1.2.0 to 1.2.1 to match package.json
  2. sarif.sh:71 — Removed broken ascii_downcase fragment anchor from helpUri; now points to check-catalog.md without fragment
  3. docs/github-actions-sarif.md:153 — Changed to jq -e '.runs[0]?.results[]?' for CI robustness
  4. docs/github-actions-sarif.md:298 — Updated example version from 1.2.0 to 1.2.1
  5. docs/github-actions-sarif.md:307 — Updated example helpUri to match code (no fragment anchor)
  6. docs/github-actions-sarif.md:377 — Changed to jq '[.runs[0]?.results[]?' for CI robustness
  7. Greptile sarif.sh:97defaultConfiguration.level was already added in round 2 (stale comment, no change needed)

@gemini-code-assist @greptile-apps — Requesting re-review.

Copy link

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces SARIF output format support via a --sarif flag, enabling integration with code scanning tools like GitHub Advanced Security. The implementation includes a new helper script sarif.sh to convert findings to SARIF format and extensive documentation in github-actions-sarif.md on how to use it with GitHub Actions. The changes in clawpinch.sh to support the new flag and refactor the UI display logic are well-done. My review focuses on the new documentation, pointing out a few areas where the examples and descriptions could be more closely aligned with the implementation to avoid user confusion.


Once uploaded, security findings appear in pull requests:

- **Code annotations**: Findings appear as inline comments on the relevant files

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

This documentation states that findings will appear as inline comments. However, the current SARIF generation implementation in scripts/helpers/sarif.sh sets the location for all findings to the repository root ("uri": ".") without specific file or line number information. This means that GitHub will likely show findings at the repository level in the 'Security' tab, but not as inline annotations on the relevant source code files. To avoid user confusion, it would be best to update this documentation to reflect the current behavior. For true inline annotations, the tool's finding data model would need to be extended to include file and line information, which could be a larger future enhancement.

"tool": {
"driver": {
"name": "ClawPinch",
"version": "1.2.1",

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The tool version in this example is hardcoded as 1.2.1. The sarif.sh script dynamically determines the version from package.json. To prevent this documentation from becoming outdated, consider using a placeholder like <clawpinch-version> and mentioning that it's dynamically populated.

Suggested change
"version": "1.2.1",
"version": "<clawpinch-version>",

Comment on lines +319 to +322
"message": {
"text": "Gateway listening on 0.0.0.0 - restricts to localhost (127.0.0.1)",
"markdown": "**Finding:** Gateway listening on 0.0.0.0\n\n**Fix:** Set gateway.host to '127.0.0.1' in openclaw.json"
}

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The example message object here doesn't quite match the structure generated by scripts/helpers/sarif.sh. The script generates a markdown message combining the finding's description and remediation fields, like (.description + "\n\n**Remediation:** " + .remediation). This example seems to use the title for the finding text. Aligning the example with the actual output would make the documentation more accurate.

Suggested change
"message": {
"text": "Gateway listening on 0.0.0.0 - restricts to localhost (127.0.0.1)",
"markdown": "**Finding:** Gateway listening on 0.0.0.0\n\n**Fix:** Set gateway.host to '127.0.0.1' in openclaw.json"
}
"message": {
"text": "Gateway listening on 0.0.0.0",
"markdown": "The gateway `bindAddress` is set to `0.0.0.0`, exposing it on all network interfaces. Any device on the local network (or the internet, if port-forwarded) can connect.\n\n**Remediation:** Set `gateway.bindAddress` to `127.0.0.1:<port>` in `openclaw.json`."
}

- Update PR results section: findings appear in Security tab as
  repository-level alerts, not as inline code annotations, because
  the SARIF location uses repository root ("uri": ".") without
  file/line info
- Replace hardcoded version "1.2.1" with placeholder
  <clawpinch-version> and note it is dynamically read from
  package.json at runtime
- Fix example message object to match sarif.sh output: "text" holds
  the short finding title, "markdown" holds description + remediation
  in the format (.description + "\n\n**Remediation:** " + .remediation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@MikeeBuilds
Copy link
Owner Author

Addressed all 3 review comments from @gemini-code-assist in commit f5a6455:

  1. Line 88 — inline annotations claim: Updated the "Viewing Results in Pull Requests" section to accurately state that findings appear in the Security tab as repository-level alerts. Added a note explaining that because ClawPinch scans runtime configurations (not source files), the SARIF output uses "uri": "." without file/line info, so findings do not appear as inline PR annotations. Also updated the introductory paragraph for consistency.

  2. Line 298 — hardcoded version: Replaced "1.2.1" with <clawpinch-version> placeholder and added a note that the version is dynamically populated from package.json at runtime.

  3. Line 322 — message object mismatch: Updated the example to match what sarif.sh actually generates:

    • "text" now shows the short finding title (.title)
    • "markdown" now shows the full description + remediation format (.description + "\n\n**Remediation:** " + .remediation)

@gemini-code-assist
Copy link

Thanks for the update, @MikeeBuilds! I've reviewed the changes in commit f5a6455 and can confirm that all three points from my previous review comments have been addressed effectively:

  1. Inline annotations claim: The documentation in docs/github-actions-sarif.md now accurately clarifies that findings appear in the Security tab as repository-level alerts, and the explanation regarding "uri": "." and the absence of inline PR annotations is clear and correct.
  2. Hardcoded version: The example in docs/github-actions-sarif.md now uses the <clawpinch-version> placeholder, with a note indicating dynamic population from package.json, which is a good improvement for maintainability.
  3. Message object mismatch: The example in docs/github-actions-sarif.md has been updated to correctly reflect the text and markdown fields as generated by sarif.sh, ensuring consistency between documentation and implementation.

Great work on these improvements!

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

3 files reviewed, 2 comments

Edit Code Review Agent Settings | Greptile

Comment on lines +101 to +103
. | [.[] | select(.severity != "ok")] | map({
"ruleId": .id,
"level": (
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SARIF includes null fields
properties.evidence and properties.auto_fix are emitted as JSON null when the input finding has those fields set to null (e.g., scanners using emit_finding ... "$evidence" with an empty/unset var typically produce empty string, but any custom scanner could output null). SARIF property values are expected to be strings/objects/arrays; null here is likely to break validation or downstream tooling. Consider normalizing with (.evidence // "") and (.auto_fix // "") (and similarly guarding .remediation comparisons against null) before constructing message.markdown and properties.

Prompt To Fix With AI
This is a comment left during a code review.
Path: scripts/helpers/sarif.sh
Line: 101:103

Comment:
**SARIF includes `null` fields**
`properties.evidence` and `properties.auto_fix` are emitted as JSON `null` when the input finding has those fields set to null (e.g., scanners using `emit_finding ... "$evidence"` with an empty/unset var typically produce empty string, but any custom scanner could output null). SARIF property values are expected to be strings/objects/arrays; `null` here is likely to break validation or downstream tooling. Consider normalizing with `(.evidence // "")` and `(.auto_fix // "")` (and similarly guarding `.remediation` comparisons against null) before constructing `message.markdown` and `properties`.


How can I resolve this? If you propose a fix, please make it concise.

Comment on lines +275 to +282
## SARIF Output Format Reference

ClawPinch produces SARIF v2.1.0 output with the following structure. The `version` field is dynamically populated from `package.json` at runtime:

```json
{
"version": "2.1.0",
"$schema": "https://raw.githubusercontent.com/oasis-tcs/sarif-spec/master/Schemata/sarif-schema-2.1.0.json",
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema URL not stable
The $schema value in the example uses raw.githubusercontent.com/oasis-tcs/.../master/..., which can change over time and may cause validator drift. Since this doc is meant to be copied verbatim into CI troubleshooting, it should reference a specific tag/commit (or omit $schema entirely) to keep the guidance deterministic.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

Prompt To Fix With AI
This is a comment left during a code review.
Path: docs/github-actions-sarif.md
Line: 275:282

Comment:
**Schema URL not stable**
The `$schema` value in the example uses `raw.githubusercontent.com/oasis-tcs/.../master/...`, which can change over time and may cause validator drift. Since this doc is meant to be copied verbatim into CI troubleshooting, it should reference a specific tag/commit (or omit `$schema` entirely) to keep the guidance deterministic.


<sub>Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!</sub>

How can I resolve this? If you propose a fix, please make it concise.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant